In this paper, we study the quantitative convergence of shallow neural networks trained via gradient descent to their associated Gaussian processes in the infinitewidth limit. While previous work has established qualitative convergence under broad settings, precise, finite-width estimates remain limited, particularly during training. We provide explicit upper bounds on the quadratic Wasserstein distance between the network output and its Gaussian approximation at any training time t 0, demonstrating polynomial decay with network width. Our results quantify how architectural parameters, such as width and input dimension, influence convergence, and how training dynamics affect the approximation error.

We have omitted a number of complicated formulas in the main text to provide clear intuition and concise proof sketch. We will list all mentioned formulas here for readers' reference. We consider the case where U = V = Aand Σ is symmetric and full-rank, and we use gradient flow. We can derive the dynamics of S = AA>as S:= (Σ S)S+ S(Σ S), which is a quadratic ordinary differential equation and it is hard to solve directly. For simplicity, define X:= X Σ 1. Then X = XΣ ΣX. (24) Solving this equation and we have And it is interesting to verify that S(t) + P(t) Σ by using the following lemma.

Neural Information Processing Systems http://nips.cc/

Bounds on the smallest eigenvalue of the neural tangent kernel (NTK) are a key ingredient in the analysis of neural network optimization and memorization. However, existing results require distributional assumptions on the data and are limited to a high-dimensional setting, where the input dimension $d_0$ scales at least logarithmically in the number of samples $n$. In this work we remove both of these requirements and instead provide bounds in terms of a measure of distance between data points: notably these bounds hold with high probability even when $d_0$ is held constant versus $n$. We prove our results through a novel application of the hemisphere transform.

While initial breakthroughs on the convergence of gradient optimization in neural networks (Li & Liang, 2018; Du et al., 2019a; Allen-Zhu et al., 2019) required unrealistic conditions on the

Stein V ariational Gradient Descent (SVGD) is a nonparametric particle-based deterministic sampling algorithm.

Neural Information Processing Systems http://nips.cc/

We thank the reviewers for their appreciative and thoughtful feedback. Reviewer 1. "However, the authors fail to bring the result to their impact of the current state OT, or any novel stochastic optimization algorithm designed to compute it faster. We will further emphasize these aspects. " A measure with 0 mean. Reviewer 2. "If the paper could show the formula for that case [TV] that would be Figure 1: Large dimensions need more samples to approximate the moments of the unbalanced optimal transport plan. Reviewer 3. ""Figure 1 illustrates the convergence"... the convergence of what?" "Figure 2 is also difficult to understand.